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Abstract 


We  report  progress  in  the  following  sireas.  (Some  of  these  results 

reflect  joint  work  with  others.) 

Set  covering  problenu.  We  developed  a  new  heuristic  based  on  a 
subgradient  approach  to  solving  the  Lagrangean  dual.  As  a 
stand-alone  heuristic  it  compares  favorably  with  others.  When 
used  to  replau:ed  the  conventional  subgradient  procedure  in  an 
exact  algorithm  based  on  branch-and-bound,  it  improves  perfor¬ 
mance  drastically. 

Problems  soluble  by  integer  programming  We  showed  that  log¬ 
ical  inference  problems  defined  by  a  “balanced”  matrix  are  solu¬ 
ble  both  by  linear  progranuning  and  by  a  nonnumeric  algorithm 
similar  to  unit  resolution.  We  showed  how  to  recognize  balanced 
0,1  matrices  in  polynomial  time,  using  gr^>h  decomposition,  and 
made  progress  toward  the  recognition  of  0,  ±1  balanced  matri¬ 
ces.  We  contributed  to  the  recognition  of  LP-soluble  set  cover¬ 
ing  problems  by  expanding  the  list  of  known  minimal  nonideal 
submatrices.  We  proved  that  the  resolution  method  of  theorem¬ 
proving  allows  one  to  check  whether  a  satisfiability  problem  can 
be  solved  as  a  linear  programming  problem  by  checking  the  same 
for  all  of  its  set  covering  subproblems. 

Inference  in  propositional  logic.  We  systematically  tested  infer¬ 
ence  algorithms  for  propositional  logic  and  identified  which  ones 
seem  to  perform  well  on  easy,  medium-difficulty,  and  hard  prob¬ 
lems,  and  to  some  extent  why  they  do.  We  developed  and  tested 
a  logic  circuit  verification  algorithm  based  on  Benders  decompo¬ 
sition  that  is  better  than  the  state  of  the  art  on  certain  classes  of 
circuits  and  worse  on  others.  We  showed  how  to  solve  the  prob¬ 
lem,  “what  are  all  the  implications  of  a  rule  base  with  respect 
to  a  given  question,”  as  a  projection  problem — completely  in  the 
Horn  case,  and  partially  in  the  non-Horn  case.  We  showed  how 
to  generate  good  (“tight”)  MILP  formulations  of  logical  rules  (in 
particular,  cardinality  rules).  We  moved  closer  to  the  goal  of 
an  efficient  solver  for  non-Horn  logic  progranuning  by  extending 
partial  instantiation  methods  to  full  first-order  logic  with  func¬ 
tion  symbols.  We  applied  logical  inference  methods  to  chemical 
engineering  design  problems. 

Inductive,  uncertainty  and  belief  logics.  We  unified  several  un¬ 
certainty  and  belief  logics  by  showing  that  they  can  be  solved 
by  a  single  linear  programming  algorithm  with  a  different  col¬ 
umn  generation  subroutine  for  each  logic.  We  showed  how  to 


2 


combine  statistical  and  boolean  methods  to  obtain  a  regression- 
based  method  for  deriving  rules  for  an  expert  system  based  on 
past  expert  behavior. 
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Introduction 


This  is  the  final  technical  report  for  research  partially  supported  by  the 
AFOSR  grant,  "Covering,  Packing  and  Logical  Inference,”  which  took  effect 
on  1  July  1991.  This  report  foUows  the  same  outline  as  the  proposal. 

The  papers  written  in  connection  with  this  grant  and  its  predecessor, 
"Mathematical  Programming  and  Logical  Inference,”  are  marked  with  an 
asterisk  in  the  bibliography.  Papers  written  under  the  present  grant  are 
marked  with  a  double  asterisk. 

2  Set  Covering  Problems 

In  a  recent  joint  paper  with  Maria  C.  Carrera,  "A  Dynamic  Subgradient- 
Based  Branch  and  Bound  Procedure  for  Set  Covering”  [5],  we  report  on  the 
development,  implementation  and  computational  testing  of  a  class  of  algo¬ 
rithms  for  the  set  covering  problem  that  has  shown  excellent  computational 
performance  on  a  large  battery  of  test  problems. 

We  address  the  set  covering  problem 

(SC)  min{cx|Ax  >  l,i  €  {0,1}”}, 

where  A  is  an  m  x  n  matrix  of  O's  and  Ts,  c  is  an  integer  n-vector,  and  1  is 
the  m- vector  of  I’s. 

Our  approach  belongs  to  a  family  of  branch  and  bound  procedures  for 
(SC)  whose  main  characteristic  is  that  at  every  node  of  the  search  tree, 
instead  of  using  the  simplex  method  to  solve  the  linear  programming  re¬ 
laxation  of  the  given  snbproblem,  it  uses  some  combination  of  primal  and 
dual  heuristics  with  subgradient  optimization  applied  to  a  Lagrangean  dual, 
possibly  incorporating  cutting  planes,  to  generate  upper  and  lower  bounds 
on  the  objective  function  value. 

Various  elements  of  this  approach  have  been  arotmd  for  about  15  years. 
Etcheberry  [25]  seems  to  have  been  first  to  use  subgradient  optimization 
instead  of  the  simplex  method.  Balas  and  Ho  [6]  tested  various  Lagrangean 
relaxations,  some  of  them  including  cutting  planes,  and  introduced  several 
primal  and  dual  heuristics  combined  with  variable  fixing  techniques,  as  weU 
as  some  new  branching  rules.  Their  algorithm  and  its  computational  perfor¬ 
mance  served  as  a  benchmark  for  subsequent  developments  in  the  80’s.  Hall 
and  Hochbaum  [27]  extended  the  approach  to  more  general  covering  prob¬ 
lems  (right  hand  side  greater  than  one).  Vasko  and  Wilson  [49]  introduced 
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an  efficient  randomized  version  of  the  greedy  heuristic.  Carrera  [15]  per¬ 
formed  a  thorough  computational  study  of  primal  heuristics,  including  the 
randomized  greedy  and  reduced  cost  heuristics.  Beasley  [10]  implemented  a 
branch  and  bound  algorithm  using  the  Balas-Ho  framework,  but  solving  the 
linear  programming  subproblems  by  the  simplex  method  and  using  some 
new  variable  fixing  rules.  In  a  later  paper,  Beasley  [11]  introduced  a  La- 
grangean  heuristic  that  generates  a  cover  after  every  subgradient  iteration. 
Fisher  and  Kedia  [26]  use  an  efficient  dual  heuristic  as  the  main  ingredient 
of  their  approach.  For  some  recent  work  on  set  covering  algorithms  based 
on  other  approaches,  see  Harche  and  Thompson  [29]  and  Nobili  and  Sassano 
[45]. 

The  centerpiece  of  the  approach  discussed  in  this  paper  is  an  integrated 
lower  bounding/upper  bounding  procedure  that  we  call  dynamic  subgradi¬ 
ent  optimization  (DYNSGRAD)  and  that  we  apply  to  a  Lagrangean  dual 
problem  at  every  node  of  the  search  tree.  This  procedure  intertwines  the 
iterations  of  subgradient  optimization,  a  lower  bounding  procedure,  with  ap¬ 
plications  of  the  reduced  cost  heuristic  (RCH),  a  primal,  i.e.  upper  bounding 
procedure  (which  however,  as  a  byproduct,  also  improves  the  lower  bound), 
followed  by  variable  fixing  applied  in  a  recursive  fashion.  Whenever  RCH 
improves  the  upper  bound,  the  current  dual  vector  (set  of  Lagrange  multi¬ 
pliers)  is  changed,  along  with  the  upper  and  lower  bounds;  and  whenever 
some  variable  is  fixed  at  1,  the  constraint  set  of  the  Lagrangean  problem  it¬ 
self  changes,  and  the  parameters  of  the  subgradient  procedure  are  adjusted: 
hence  the  qualifier  "dynamic”  in  the  name  of  the  procedure. 

Other  new  features  of  our  algorithm  include  some  primal  and  dual  heuris¬ 
tics,  a  recursive  variable  fixing  procedure,  and  new  branching  rules. 

0\ir  procedures  have  been  extensively  tested,  both  on  randomly  gener¬ 
ated  and  on  real  world  problems.  As  a  stand-alone  heuristic,  the  dynamic 
subgradient  procedure  compares  favorably  with  all  other  heuristics  known 
to  us  in  terms  of  the  quality  of  solutions  obtainable  with  a  certain  compu¬ 
tational  effort.  As  a  new  ingredient  of  our  branch  and  bound  procedure,  it 
has  drastically  improved  the  performance  of  the  latter  as  compared  to  its 
older  version  which  uses  at  every  node  the  standard  subgradient  optimiza¬ 
tion  procedure.  It  also  compares  favorably  with  other  branch  and  bound 
procedure. 
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3  Recognizing  Packing,  Covering  and  Logical  In¬ 
ference  Problems  with  an  Integral  Underlying 
Polyhedron 

3.1  A  Class  of  Logic  Problems  Solvable  by  Linear  Program¬ 
ming 

In  propositional  logir>  several  problems  such  as  satisfiability,  MAXSAT  and 
logical  inference,  can  be  formulated  as  integer  programs.  For  example,  given 
a  set  5  of  clauses  and  a  weight  vector  to  whose  components  are  indexed  by 
the  clauses  in  5,  the  weighted  maximum  satisfiability  problem  (MAXSAT) 
consists  in  finding  a  truth  assignment  that  maximises  the  total  weight  of 
the  satisfied  clauses.  MAXSAT  can  be  formulated  as  the  integer  program 

HiLl 

Ax  +  s>  1-  n(A) 

x€{0,ir,s€{0,ir 

where  A  is  a  0,  ±1  matrix. 

The  above  three  problems  are  NP-hard  in  general  but  SAT  and  logical 
inference  can  be  solved  efficiently  for  Horn  clauses,  clauses  with  at  most  two 
literals  and  several  related  classes  [17], [48].  MAXSAT  remains  NP-hard  for 
Horn  clauses  with  at  most  two  literals. 

In  [19],  we  show  that  the  above  three  problems  can  be  solved  in  polyno¬ 
mial  time,  as  linear  programs,  whenever  the  matrix  A  associated  with  S  is 
a  balanced  0,±1  matrix  as  defined  by  IVuemper  [47],  see  also  [46].  A  0,±1 
matrix  A  is  balanced  if,  in  every  submatrix  of  A  with  exactly  two  nonzero 
entries  per  row  and  per  column,  the  sum  of  the  entries  is  a  multiple  of  4. 

The  key  to  obtmning  this  result  is  the  following  theorem.  If  A  is  a 
balanced  0,±1,  then  R(A)  =  €  TZ"'*'"*  :  Ax  -I-  s  >  1  -  n(A),  0  < 

x,s  <  9I}  is  an  integer  polytope. 

3.2  Recognizing  balanced  0, 1  matrices 

Our  approach  to  recognizing  balanced  matrices  uses  graph  decomposition. 
To  a  0, 1  matrix  A  we  associate  a  bipartite  graph  G{A)  as  follows:  the  node 
set  of  G(A)  is  partitioned  into  V  and  V®  which  represent  the  row  set  and 
the  column  set  of  A,  and  G(A)  has  an  edge  connecting  node  i  and  node  j 
if  and  only  if  Oij  =  1.  A  0, 1  matrix  is  balanced  if  and  only  if  the  length  of 
every  chordless  cycle  in  the  associated  bipartite  graph  is  a  multiple  of  4. 
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A  decomposition  theorem  is  established  in  [20]  Parts  11- VI  and  used  in 
Part  VII  to  recognize  in  polynomial  time  whether  a  0, 1  matrix  is  balanced. 
A  2-jotnin  a  bipartite  graph  G  is  aset  of  edges  E\UE^  such  that, for  t  =  1,2, 
Ei  induces  a  complete  bipartite  subgraph  G,-  of  G,  the  graphs  Gi  and  Gs 
have  disjoint  node  sets  and  G  \  (£i  U  £3)  is  disconnected.  A  double  star 
cutset  in  a  bipartite  graph  G  is  a  node  set  S  such  that  G\S  is  disconnected 
and  there  exist  two  adjacent  nodes  it,  v  with  the  property  that  every  node  of 
S  is  adjacent  to  u  or  v.  Restricted  balanced  matrices  are  those  where  every 
cycle  of  G(A)  has  length  congruent  to  0  modulo  4.  Yannakakis  [50]  showed 
how  to  recognize  them  in  linear  time. 

Our  decomposition  theorem  states  that,  if  a  balanced  bipartite  graph  is 
not  restricted  balanced,  then  it  contains  a  2-join  or  a  double  star  cutset. 

Double  star  cutsets  and  2-joins  can  be  used  recursively  to  decompose  a 
given  bipartite  graph  G  into  elementary  blocks  that  contain  neither.  In  [20], 
it  is  shown  how  to  perform  this  decomposition  so  that  only  a  polynomial 
number  of  elementary  blocks  are  created  and  G  is  balanced  if  and  only  if  all 
the  elementary  blocks  are  restricted  balanced.  A  polynomial  algorithm  for 
recognizing  balanced  0, 1  matrices  follows. 

The  decomposition  theorem  for  balanced  0, 1  matrices  has  been  recently 
extended  to  balanced  0,±1  matrices  in  [21]  but  a  new  decomposition  is 
required  as  well  as  a  new  elementary  block  which  is  not  restricted  balanced. 
The  algorithmic  issues  are  currently  under  investigation. 

3.3  Ideal  Matrices 

A  0,1  matrix  M  is  ideal  if  all  vertices  of  the  set  covering  polyhedron 
{x  :  Mx  >  1,  X  >  0}  have  only  0,1  components.  The  0,1  matrix  A  is 
said  to  be  minimally  nonideal  if  the  linear  system  Ax  >  1  has  no  redun¬ 
dant  constraints,  the  polyhedron  P{A)  has  a  fractional  extreme  point  but 
all  the  polyhedra  obtained  by  intersecting  P{A)  with  one  of  the  hyperplanes 
Xj  =  0  or  Xj  =  1  for  j  =  1, . . ., n  are  integral.  Minimally  nonideal  matrices 
are  important  since  they  represent  the  fundamental  violators  of  idealness, 
i.e.  every  nonideal  matrix  must  contaun  a  minimally  nonideal  matrix  .  The 
study  of  minimally  nonideal  matrices  is  the  counterpart  for  the  set  cover¬ 
ing  problem  of  the  study  of  minimally  imperfect  matrices  for  the  set  packing 
problem.  In  [23],  we  expand  the  list  of  known  minor  minima]  nonideal  matri¬ 
ces  by  several  hundred.  Many  of  these  examples  are  obtained  polyhedrally, 
by  constructing  new  minimally  nonideal  matrices  from  old  ones.  We  present 
a  conjecture  that  might  be  viewed  as  the  counterpart  for  ideal  matrices 
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of  Berge’s  Strong  Perfect  Graph  Conjecture.  We  provide  evidence  for  the 
conjecture  by  completely  characterizing  aD  minimally  nonideal  circulants. 

3.4  Resolution  and  Integrality 

J.  Hooker  identified  another  class  of  inference  problems  that  can  be  solved  by 
linear  programming  [39].  If  resolution  is  applied  to  a  satisfiability  problem, 
then  it  describes  an  integral  poly  tope  (and  is  therefore  soluble  by  LP)  if  and 
only  if  its  set  covering  subproblems  do.  A  coroUary  of  this  work  is  that  the 
facets  of  the  ssatisfiability  polytope  consists  of  facets  of  various  set  covering 
poly  topes  plus  prime  implications  obtained  by  resolution. 

4  Inference  in  Propositional  Logic 

4.1  Computational  Testing  of  Satisfiability  Algorithms 

F.  Harche,  J.  Hooker  and  G.  Thompson  did  a  computational  study  of  sev- 
era]  satisfiability  algorithms  based  on  tree  search,  including  versions  of  the 
Davis-Putnam-Loveland,  Jeroslow-Wang,  Horn  relaxation,  branch-and-cut, 
and  column  subtraction  algorithms  [28].  All  testing  was  done  on  the  same 
machine  using  as  many  common  subroutines  as  possible.  The  problems  were 
tested  on  a  large  set  of  benchmark  problems  collected  by  F.  J.  Radermacher. 
It  was  found  that  methods  that  employ  weak  relaxations  at  the  search  tree 
nodes,  such  as  the  Horn  relaxation  method,  are  much  more  effective  for  the 
easier  problems  and  much  less  effective  for  the  harder  problems.  The  column 
subtraction  method  was  the  most  robust,  as  it  was  the  only  method  capable 
of  solving  all  the  problems  in  the  set.  The  Jeroslow-Wang,  branch-and-cut 
and  one  version  of  the  Davis-Putnam-Loveland  are  fastest  on  problems  of 
intermediate  difficulty. 

Based  partly  on  this  experience,  J.  Hooker  developed  a  fast  algorithm  for 
the  incremental  satisfiability  problem  [36].  This  is  the  problem  of  checking 
whether  a  satisfiable  set  of  propositions  remains  satisfiable  when  another 
proposition  is  added.  It  is  a  key  subproblem  for  lo^c  circuit  verification  [42] 
and  partial  instantiation  methods  for  first  order  predicate  logic  [35].  Com¬ 
putational  study  showed  that  in  most  cases,  using  the  incremental  algorithm 
is  an  order  of  magnitude  faster  than  re-solving  the  problem  from  scratch. 
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4.2  Logic  Circuit  Testing 

J.  Hooker  and  H.  Yan  developed  and  tested  a  new  algorithm  for  Ic^c  circuit 
verification,  which  is  based  on  Benders  decomposition.  This  is  a  problem  on 
which  Japanese  and  American  industry  has  worked  intensively  for  several 
years.  They  compared  the  performance  of  their  prototype  algorithm  with 
the  most  recent  state-of-the-art  code  (KBDD). 

The  Benders  code  ran  more  slowly  than  KBDD  on  most  of  the  stan¬ 
dard  benchmark  problems  in  the  literature.  Hooker  and  Yan  attributed  this 
to  the  large  large  number  of  exclusive-OR  gates  in  these  problems.  They 
generated  random  problems  without  exclusive-OR  gates  and  found  that  the 
Benders  approach  is  an  order  of  magnitude  faster  than  KBDD  on  almost 
all  of  them.  They  conclude  that  the  Benders  approach  is  very  effective  on 
certain  classes  of  problems  and  ineffective  on  others,  and  its  strengths  are 
somewhat  complementary  to  those  of  BDD  algorithms. 

4.3  Inference  as  Projection 

The  problem  of  deriving  all  Inferences  in  propositional  lofpc  that  answer  a 
given  question  (i.e.,  that  contain  a  given  set  of  atomic  propositions)  is  a 
type  of  projection  problem.  J.  Hooker  wrote  a  paper  [37]  that  shows  that 
solving  a  related  polyhedral  projection  problem  is  equivalent  to  applying  a 
form  of  unit  resolution  to  the  logical  problem.  This  provides  a  practical 
algorithm  for  solving  Horn  projection  problems  and  partially  solving  non- 
Hom  problems. 

4.4  Formulation  of  Logic  Constraints. 

J.  Hooker  and  H.  Yan  wrote  a  paper  on  tight  formulations  of  cardinality 
rules  [43].  These  results  show  how  to  get  good  representations  of  complex 
logical  constraints,  such  as  a  rule  base,  in  a  mixed  integer  programming 
model. 


4.5  Logic  Programming 

Current  logic  programming  systems  (PROLOG,  PROLOG  III  etc.)  pro¬ 
vide  inference  methods  that  at  best  are  complete  only  for  datalog  langu^es 
(universally  quantified  first-order  Horn  logic).  J.  Hooker  and  G.  Rago  [41] 
extended  the  partial  instantiation  approach  of  R.  Jeroslow  to  accommodate 
function  symbols,  so  that  it  becomes  applicable  to  full  non-Horn  first-order 
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logic.  To  avoid  the  problem  of  undeciability,  they  introduced  a  device  that 
checks  whether  a  logic  program  has  a  model  that  involves  a  given  maxi¬ 
mum  level  of  function  nesting,  which  is  decidable.  Work  to  date  lays  the 
theoretical  foundations. 

4.6  Logic-Based  Methods  for  Design 

J.  Hooker,  H.  Yan,  I.  Grossmann,  and  R.  Raman  wrote  a  paper  [44]  that 
shows  how  logic  cuts  can  be  used  to  speed  the  solution  of  MILP  models  for 
chemical  process  design  problems. 

5  Inference  Methods  for  Belief  Systems 

5.1  A  Linear  Programming  Framework  for  Combining  Evi¬ 
dence. 

Andersen  and  Hooker  reviewed  the  AI  literature  on  evidential  reasoning  and 
discovered  that  many  uncertainty  logics  fit  the  same  LP  model.  They  show 
in  a  new  paper  [3]  how  to  implement  several  logics  in  a  single  LP  framework 
by  "plugging  in”  different  column  generation  subroutines  for  the  different 
logics.  In  particular  they  formulated  two  practical  logics  that  account  for 
second  order  probabilities  (i.e.,  the  reliabilities  of  the  sources  of  probability 
information). 

J.  Hooker  also  published  a  survey  paper  on  mathematical  programming 
methods  for  reasoning  under  uncertainty  [38]. 

5.2  Obtaining  Rules  for  Expert  Systems 

E.  Boros,  P.  Hammer  and  J.  Hooker  showed  how  an  approach  analogous 
to  statistical  regression  can  be  used  to  generate  rules  for  an  expert  system, 
based  on  the  past  decisions  of  experts.  It  has  the  advantage  that  statistical 
tests  can  be  used  to  decide  whether  the  derived  rules  are  statistically  valid. 

They  also  moved  partway  toward  extending  this  work  to  the  nonboolean 
case.  In  a  second  paper  [13],  they  show  how  a  network  flow  model  can  be 
used  to  derive  the  best-fitting  (nonboolean)  rules  when  they  are  required  to 
be  “monotone.” 
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